Multimodal learning of words: A study on the use of speech synthesis to reinforce written text in L2 language learning

Past research has shown that the use of multimedia, such as pictures, audio narration, and video, can be beneficial in computer aided instruction. We propose that spoken words generated by speech synthesis can be used to reinforce written text during L2 language instruction, and can lead to a more robust learning experience than providing written language input alone. Two in-vivo studies were conducted with ESL (English as a second language) students to investigate the effect of providing spoken language produced by speech synthesis during different instructional events in REAP, a computer based vocabulary tutor. Our results show that students benefit from spoken language input, particularly when they are strongly encouraged to listen to words. Furthermore, our studies seem to suggest that on demand English text-to-speech synthesis may be good enough to provide added value during computer based L2 language instruction.